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CHARTER SCHOOLS ARE PUBLIC SCHOOLS 

that operate outside school district 
control and most state regulations. Their 
relative autonomy lets them determine 
their own instructional program. Yet this 
independence is not entirely without 
limitation. Charter schools operate 
under a five-year, renewable contract 
(“charter”) negotiated with a district, 
county office of education, or the State 
Board of Education. They must also 
attract students to open and retain 
students to remain viable. Groups can 
start schools from scratch or convert 
existing schools to charter status. 
Whether a charter school is a “start-up” 
or “conversion,” students do not have to 
enroll but choose to. 

In addition to enrollment demands, 
charter schools in California are 
expected to demonstrate adequate or 
improved student performance under 
federal and state accountability systems 
just like noncharter public schools. 
A charter school also faces specific 
performance requirements under state 
law and must meet performance goals 
outlined in its charter. If it does not 
meet those goals, the charter-granting 



agency can revoke or refuse to renew the 
school’s charter.’ 

Despite these limitations, charter 
schools’ instructional program can be 
more innovative and flexible than 
district-run schools. In theory, this free- 
dom should lead to better student 
performance because a charter school 
can create a program that better fits the 
students and teachers who choose to be 
at the school. In fact, many charters are 
formed with a specific type of student or 
program in mind. 

California’s first charter school 
opened in 1993. Since then, the 
numbers have grown steadily each year. 
Parents want to know if these schools 
are providing sound instruction, and 
local and state policymakers care about 
whether allowing schools more inde- 
pendence translates into higher student 
achievement. 

This report addresses two critical 
questions: 

• How does the academic performance 
of California charter schools differ 
from that of noncharters? 

• How does the academic performance of 
different kinds of charter schools vary? 



The report examines school and 
student performance as measured by state 
tests using a research method — statistical 
regression — that makes it possible to 
reasonably compare groups of schools 
that serve somewhat different students. 
Charter elementary, middle, and high 
schools are each compared to their 
noncharter counterparts. In addition, the 
report compares the performance of 
various subgroups of charter schools, 
including start-ups versus conversions, 
those that operate as more traditional 
classroom-based schools versus 
nonclassroom-based, and those that are 
operated by a school management organ- 
ization versus charter schools that are not. 

This report also compares schools 
based on the measures used by the state 
for its accountability system, which 
includes statewide ranks, similar schools 
ranks, and success meeting growth targets. 

This report begins with a description of 
the analytic approach, the set of schools 
studied, and the measures used to gauge 
performance. This information defines and 
clarifies the terms and figures that follow in 
the performance comparisons that begin 
on page 8. 
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Charter Schools in California: Vital Statistics 




Academic Year 

Enrollmentf 10.761 23,228 30,977 39,624 48,101 68,685 99,048 113,956 132,643 158,942 167,764 181,818 202,683 

% of State’s 

Enrollment 0.2% 0.4% 0.6% 0.7% 0.8% 1.2% 1.7% 1.9% 2.2% 2.5% 2.7% 2.9% 3.2% 

Note: To be counted among the schools open in a given year, a school must operate at a minimum from Nov. 1 through February, 
t Enrollment data is not available for a few schools each year. 



The number of charter schools in California has grown significantly since 1992 
when they were first authorized by the state. A total of 574 charter schools oper- 
ated throughout California in 2005-06 (the most current data available), 
representing about 6% of all state public schools that year. Because charters tend 
to have fewer students than “regular” public schools, they enrolled only 3.2% of 
the state’s students. These are small increases from 2004-05 when the state 
ranked 12th in the nation in the percent of public schools that were charters and 
11th in the percent of students enrolled in charters. (The national statistics 
include the 50 states and the District of Columbia.) 

Among the 1,034 school districts and county offices of education in California, 248 
(24%) had at least one charter school. Eight districts-which together enroll about 6,300 
students-have converted all their schools to charter status. Five of those all-charter 
districts have only one school. The other three districts have two, three, and five schools. 



Compared to California's noncharter public schools, charter schools are less likely to be 
elementary schools and more likely to be high schools. Among the 9,001 noncharteis open 
in 2005-06, 62% were elementary, 15% were middle, and 23% were high schools. In 
contrast, the 574 charters consisted of 47% elementary, 12% middle, and 41% high. 

Taken as a whole, the state's charters also differ from noncharters in the students 
they sen/e and the teachers they employ. Charters sen/e greater proportions of 
African American and white students and smaller percentages of Asians, Latinos, 
and English learners. (It is possible that differences in the percentages of English 
learners are due to differences in ability to help students achieve proficiency in 
English. However, publicly available state data do not allow for investigation into 
this question.) In addition, charters have higher percentages of teachers who are 
not fully credentialed and have two years or less of experience. The table below 
shows the most recent data available. 



Percentages for All Schools Combined* 



Students 


Charters 
(574 schools) 


Noncharters 
(9,001 schools) 


African American 


12% 


8% 


Asian 


4% 


8% 


Latino 


38% 


48% 


White 


39% 


30% 


English Learner 


17% 


25% 


Teachers 


Teachers Not Fully Credentialed 


15% 


5% 


Teachers with Two Years Experience or Less 


24% 


12% 



* If all charter schools were combined into one school and all noncharter schools were combined into another school, these are the percentages that would result. The percentages related 
to student ethnicities do not sum to 100% because not all ethnicities are reflected in the table. 



Data; California Department of Education (CDE) 
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How to Interpret the Findings in this Report 

This report adjusts for differences in important student characteristics when comparing performance outcomes 



This report relies on a number of 
techniques to make performance 
comparisons more meaningful. It uses a 
statistical technique to try to strip away 
the effects of student and teacher 
characteristics and school size on 
performance. It also looks at perform- 
ance in past years to bolster claims about 
performance in 2006 and examines the 
consistency of results across multiple — 
albeit overlapping — measures. Further, 
the report focuses on subsets of schools 
that have data on all those performance 
measures. Finally, analyses are generally 
broken down by school type — elemen- 
tary, middle, and high — to compare like 
with like. 

Three concepts guide the analytic approach 

Three concepts are at the fore of this 
year’s analytic approach: controlling for 
student characteristics, showing data 
from multiple years, and triangulating 
findings.^ 



Controlling lor student characteristics — 
"validity” 

This study uses two variables to statisti- 
cally control for differences in schools: 
the size of enrollment and the School 
Characteristics Index (SCI). 

Students’ academic performance is 
strongly associated with their back- 
grounds, in particular their parents’ 
education and socioeconomic status. 
Therefore, to evaluate the performance 
of a school, it is important to take into 
consideration these characteristics. 



over which the school staff normally 
has no influence. 

This report examines performance 
differences between groups of schools 
after applying statistical controls for 
specific school characteristics. As a result, 
it provides a more valid assessment of 
schools’ performance than would a 
comparison that did not take these char- 
acteristics into account. 

The research team controlled for the 
first variable — size of enrollment — 
because in many settings there is 
evidence of a link between school size 
and student achievement, and charters 
and noncharters differ substantially in 
their average enrollments. The second 
variable — the SCI — is released annually 
as part of the Base API report. A 
school’s SCI value, which can range 
roughly from 100 to 200, summarizes 
its student demographics and school 
and teacher characteristics, as they are 
associated with academic performance. 



Fligher SCI values are associated with 
less student poverty, higher parental 
education levels, etc., and higher 
expected academic performance. In this 
report, school characteristics, SCI, and 
student characteristics are used inter- 
changeably. (For more on the SCI, see 
the box on page 5.) 

As important as the SCI factors are, 
they still do not cover all the variables that 
might possibly contribute to differences 
in school performance or student achieve- 
ment. One critical factor not accounted 



for is the motivation of parents and 
students. Parents who elect to send their 
children to charter schools instead of the 
neighborhood public school may have a 
greater stake in their children’s education, 
and that can have an effect on student 
achievement that an analysis such as this 
simply cannot measure. A second factor is 
funding — how much a school gets and 
how it spends it. The complexities of 
determining the resources available to 
charters and noncharters, as well as the 
expenditure constraints and opportuni- 
ties that different types of charters have, 
are beyond the scope of this report and 
are not considered. 

In the end, the research technique used in this 
study can indicate d^erences that are not attribut- 
able to SCI factors and school size. But it cannot 
provide certainty that differences in achievement 
between charters and other public schools are due 
only to a school’s status as a charter. 

Showing data from multiple years — 
"stability” 

Findings that hold consistently across 
multiple years are stronger and more 
credible than those that are more short- 
lived. This report replicates the main 
2006 analyses with data from previous 
years and notes whether the results have 
been stable over time.^ 

This approach is not the same as a 
longitudinal analysis, which tracks the 
same students over time. Such an analysis 
is not currently possible with California’s 
publicly available data. 

Triangulating findings — "consistency” 
Findings that are consistent across meas- 
ures, as well as over time, are more robust 
and defensible. This study reports 
school-level results from multiple meas- 
ures — Academic Performance Index 
(API), adequate yearly progress (AYP), 



The School Characteristics Index (SCI) is primarily a summary of students’ backgrounds. 

A low SCI value means the students face a high level of challenge. 
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The School Characteristics Index (SCI) is a powerful “control variable” 

The state has created a composite index, the Schooi Characteristics Index (SCi), to summarize muitipie factors that are associated with student perform- 
ance on state tests but are iargeiy beyond the controi of the schoois themseives. When comparing the performance of groups of schoois, this study 
“controisfor”schoois’SCi vaiues to adjust for differences in these important factors.This makes it possibie to estimate how one group’s performance wouid 
compare to another group’s if they had simiiar students and teachers. The use of the SCi prompts two questions: what factors does the SCi inciude, and 
how strong is it as a controi variabie? 



The Schoois Characteristics Index includes the following factors: 

• Student ethnicity: percent in each of seven ethnic categories; 

• Average parental education level; 

• Percent of English learners; 

• Percent of students with disabilities; 

• Percent of students in the Gifted and Talented Education 
program (GATE); 

• Percent of students who have been reclassified from 
“English learner’’ to “fluent English proficient (RFEP);”'* 

• Percent of migrant education students; 

¥ Students who have been reclassified have the same impact as English learners on a school’s SCI. 

What makes something a strong “statistical control”? 

As a group, charter schools perform somewhat differently than noncharter 
schools. Likewise, there are differences among different types of charter 
schools. As noted elsewhere, the goal of this analysis is to see how much of that 
difference is based on a school’s status as a charter school or a particular type 
of charter. To inform that question, the study controls for differences in 
schools’ SCI values and another factor believed to have some relationship 
to school performance-school size. 



• Percent of students in the free/reduced-priced meals program; 

• Percent of fully credentialed teachers; 

• Percent of teachers with emergency permits; 

• Average class size; 

• Student mobility: percent of students enrolled since the beginning of the 
school year; 

• Whether the school operates a multitrack, year-round educational 
program; and 

• Percent of enrollment in various grade spans. 

■Thus, a school does not suffer a “penalty” for redesignating an English learner. 

The table below shows the proportion of variation in school performance 
that is explained by the combination of SCI and school size. Statisticians 
refer to this proportion of variation as “R^’’ and the closer “R^” is to 100%, 
the more the model explains any differences. As the data show, the R^ in the 
models is high, ranging from 64% to 87% depending on the outcome 
measure and grade span. (The SCI accounts for the vast majority of that 
variation.) The data below are for the charter-versus-noncharter analyses. 



SCI and School Size Explain a Large Proportion of Variation (R^) in 2006Anaiyses of Charters Versus Noncharters 



Proportion of Explained Variation-R^-in Models Including Only SCI and Size 



Outcome Measure 


Elementary 


Middle 


High 


Base API 


83% 


85% 


80% 


AYP English 


84% 


87% 


76% 


AYP Math 


73% 


75% 


75% 


CST English 


78% 


83% 


77% 


CST Math 


64% 


75% 


N/A 


CAHSEE English 


N/A 


N/A 


82% 


CAHSEE Math 


N/A 


N/A 


80% 



The large proportion of variation explained by SCI and size increases the confidence in this study’s estimates of the effect of charter status (or being a 
specific type of charter school) on academic performance. 

See www.cde.ca.gOv/ta/ac/ap/documents/tdgreport0400.pdf for the technical foundation of the School Characteristics Index (SCI), or see 

www.cde.ca.gov/ta/ac/ap/documents/simschl06b.pdf for a less technical summary. 



© Copyright Z007 by EdSource, Inc. 



June 2007 • California Charter School Performance • 5 







EDSOURCE REPORT 



California Standards Tests (CSTs), and 
California High School Exit Exam 
(CAHSEE) — to examine the consis- 
tency of findings. 

Granted, some of these measures 
substantially overlap each other. 
For example, AYP and API results are dif- 
ferent ways of “packaging” CST (and, 
for high schools, CAHSEE) scores. 
However, each of the measures reported 
here is important to educators and poli- 
cymakers in its own right because each 
gives a different look at performance. 
For example, AYP results show the 
percentage of students meeting the 
federal and state goal of proficiency on 
state content standards, but API scores 
indicate the distribution of scores across 
the entire spectrum of performance 
levels on the CSTs. And reporting 
CST results from single benchmark 
grades helps create “apples to apples” 



comparisons of schools with different 
grade configurations. 

This report reflects multiple performance 
analyses on a specific subset of schools 

This study includes only schools with 
2006 data on API, AYP, grade-specific 
CST, and for high schools, CAHSEE 
results.'* Altogether 346 charter schools 
and 7,122 traditional public schools 
met that criteria. This means that 40% 
of the 574 charter schools and 21% of 
the 9,001 noncharter schools open in 
2005—06 are not represented in this 
study. For both sets of schools, high 
schools are much more likely to be 
excluded than elementary and middle 
schools. Among both charters and non- 
charters, the percentage of excluded 
high schools is the same — 53%. 

The vast majority of excluded 
schools — about 95% of both charters and 



noncharters — lack API and/ or SCI data. 
For charters, the main reason for not 
having API and/or SCI data is that the 
schools are “small” (have 1 1—99 test scores) 
and therefore are not assigned SCI values. 
Noncharters have two main reasons for 
lacking API/SCI data: they are too small 
or they are held accountable under Cali- 
fornia’s Alternative Schools Accountability 
Model (ASAM), which does not issue SCI 
values.^ The excluded high schools tend to 
have very high rates of student mobility, 
which may lower the number of valid 
student test scores to the point that the 
school is deemed “small” and therefore 
does not receive an API and/or SCI score. 

This Study categorizes schools in multiple ways 

The report compares charter schools at 
each grade span to their noncharter 
counterparts, a reasonable approach 
given that performance data are gener- 
ally tracked and reported based on these 
three grade-level divisions. Schools are 
classified as elementary, middle, or high 
based on their designation for the API.^ 
In addition to dividing schools by 
grade span, the analyses break charters 
into some major categories: 



This study includes only schools with 2006 data on API, AYP grade-specific CST, 

and for high schoois, CAHSEE results. 



figure 1 Count of Charters in this Analysis, Cross-tabulated by Grade Level and Type 



Member of Management 
Organization or Nonmember 


Classroom-based or 
Nonclassroom-based 


Conversion 
or Start-up 


Elementary 


Middle 


High 


Total 


Member School 


Classroom-based 


Conversion 


4 


1 


0 


5 


Start-up 


14 


13 


15 


42 




Nonclassroom-based 


Conversion 


0 


0 


0 


0 




Start-up 


6 


0 


6 


12 


Nonmember School 


Classroom-based 


Conversion 


54 


12 


4 


70 


Start-up 


84 


27 


49 


160 




Nonclassroom-based 


Conversion 


4 


0 


5 


9 




Start-up 


17 


1 


30 


48 


Totai 


183 


54 


109 


346 



Data: California Department of Education EdSource 6/07 
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• those run by a management organiza- 
tion versus those not run by such an 
organization; 

• conversions versus start-ups; and 

• classroom-based versus nonclassroom- 
based schools. 

(Those terms are defined in the rele- 
vant sections that follow.) Figure I on 
page 6 provides the counts of charter 
schools in each subcategory. 

This report looks at multiple performance 
measures 

As previously described, this study 
compares the performance of groups of 
schools on various measures. Specifi- 
cally, the analyses that follow use these 
test score results: 

• The Academic Performance Index 
(API) annual school-level measure. 
The API reflects scores from Califor- 
nia Standards Tests (CSTs)^ in 
English, math, social science (for 
middle and high schools only), and 
science; a norm-referenced test® for 
grades 3 and 7; and the California 
High School Exit Exam (CAHSEE). 

• Annual measurable objectives for 
adequate yearly progress (AYP), as 
summarized by percent of students 
scoring proficient or above on CSTs 
in English language arts and math. 
Eor high schools, percent proficient 
is based on grade 10 results on 
the CAHSEE. 

• Mean scale score on the CSTs at key 
grades and for two subjects. The analy- 
sis looks at grades 3 and 7 for English 
and math, as well as grade 10 for 
English.^ (Scale scores take into 
account the difficulty of test ques- 
tions, allowing scores to be added, 
averaged, or otherwise aggregated.) 

• Mean scale score of 1 0th graders on 
the CAHSEE in English language 
arts and math. 



Two concepts are key to understanding the performance results 

Two concepts are important to understand when viewing the performance comparisons that foiiow: 
statistical significance and effect size. 

A resuit is statisticaiiy significant when the anaiysis shows it is probabiy not due to chance varia- 
tion aione. (Think of a coin being flipped 100 times. One wouid expect “heads” to come up 50 
times, but chance variation may produce, say, 45 or 55 heads, if instead heads came up 30 or 70 
times, however, one wouid suspect that the coin was weighted to one side or the other.) 

Researchers vary somewhat in the threshoid they use for statisticai significance. This study foiiows 
one common practice, reporting statisticai significance at three ieveis: 0.10, 0.05, and 0.01.“ 
Those ieveis indicate the chance that a resuit is due to random variation. For exampie, a resuit that 
is statisticaiiy significant at the 0.05 ievei means that there is a 5% chance that the resuit is due 
to random variation. 

Statisticai significance gives information about the iikeiihood of a resuit, but it does not indicate 
the size of an effect. For exampie, it does not teii whether a difference of four API points is iarge, 
moderate, smaii, or negiigibie. 

Another measure-effect size-heips to interpret the magnitude of the resuits, it puts performance 
comparisons in reiation to the variation of performance of each group.'Hhe most common guide- 
iines for interpreting effect size propose vaiues of 0.20, 0.50, and 0.80 to represent, respectiveiy, 
smaii, moderate, and iarge effects. 

Effect size aiso piaces resuits from various measures such as APi and AYP on a common scaie. For 
exampie, the performance comparison of charter middie schoois and noncharter middie schoois 
on pages 10-11 indicates that, after adjusting for differences in student characteristics and schooi 
size, charters scored on average 41 points higher than noncharters. It also shows that charters 
outperformed noncharters on the percentage of students scoring proficient or above on the Cali- 
fornia Standards Test in English by nine percentage points. Differences of 41 API points and nine 
percentage points are not on the same scale, but the effect sizes-respectively, 0.44 and 0.41-are 
on the same scale and indicate that the effect of being a charter middle school on API scores and 
the percent proficient and above on the English CST are about the same. In this case, the effect is 
small to moderate, based on the guidelines described above. 
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Findings from Analyses that Control for Student Characteristics 
and Enrollment 

On pages 8 through 13, the results based on school level (elementary, middle, and high) are reported. The findings on charters run by management 
organizations are explained on pages 14-15, and the results that compare conversions with start-ups are on pages 16-1 7. And on pages 18-19, 
the findings based on comparing classroom-based versus nonclassroom-based charter schools are detailed. Note that the performance analyses in 
this section are based on the subset of schools that have data for the measures described on pages 4 and 6. 



With student characteristics accounted for, eiementary charter schoois did not perform 
as weii as eiementary noncharters 



Based on the subset of schools that have the 
necessary data, this analysis looks at 183 
charter elementary schools that served nearly 
78,000 students and 4,965 noncharter 
elementary schools that enrolled more than 
2.9 million students. Most of these charter ele- 
mentary schools (85%) are classroom-based. 
Two-thirds are start-ups. 

The median charter elementary school in this 
analysis served fewer disadvantaged students 



than the median noncharter elementary school. 
For example, in the median charter, 4% of stu- 
dents had parents who did not graduate from 
high school, but 15% of students in the median 
noncharter had parents with that level of educa- 
tion. On the other hand, charter students were 
considerably more likely to have an inexperi- 
enced teacher: the median charter had 20% of 
teachers with two years of experience or less, 
and the median noncharter had 9%. Charter 



elementary schools enjoyed a slightly lower 
challenge level overall, as indicated by a slightly 
higher median SCI value: 172.6 versus 169.7 for 
noncharters. (Based on SCI values, the median 
charter elementary school would be predicted 
to score 23 points higher on the API than the 
median noncharter elementary school.) It is 
important to "control for” those differences 
between the two types of elementary schools 
when comparing their performance. 



2005-06 Enroriment for Charter and 
Noncharter Erementary Schoors Included in this Analysis 



1,400- 

1 , 200 - 

1 , 000 - 

800- 

600- 

400- 

200 - 

0 



91 Upper Outliers 



1 0 Upper Outliers 

^ 939 



1,164 



Median; S64 students 




0 Lower Outliers 



Median; 329 students 
126 



0 Lower Outliers 



“I” 



Charters (183 schools) Noncharters (4,965 schools) 



2006 Schools Characteristics Index (SCI) Values for Charter 
and Noncharter Elementary Schools Included in this Analysis 




The rectangular boxes on the chart show the range from the 25th percentile (the bottom of the box) to the 75th percentile (the top of the box). The line Inside the box marks the 50th percentile or the median (half 
the schools are above this line and half are below). The small horizontal lines above and below the box mark the beginning of the "outliets'-those schools with enrollments (left chart) or with SCI values (right chart) 
far from the median. 
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After controlling for differences in enrollment and student characteristics, 
charter elementary schools scored about nine points lower than noncharter 

elementary schools on the 2006 Base API. 



After controlling for differences in enroll- 
ment and school characteristics, charter 
elementary schools scored lower than non- 
charter elementary schools on the API. The 
effect is -8.8 API points, or 10% of a standard 
deviation. 

To put that difference in context, consi- 
der that the average API of an elementary 
noncharter is 766, which is toward the 
bottom of Decile 6 on the API. The expected 
performance of a hypothetical "average” char- 
ter elementary school that could somehow 



enroll the same number of students with the 
same demographics and employ teachers 
with the same experience and credential 
levels would be 757, just at the cut point of 
API Deciles 5 and 6. This effect is not large, 
but it is fairly consistent with results using 
other measures. Notably, charters’ 2006 
math performance trailed noncharters’ by a 
larger margin than on the API (effect size of 
-0.26 whether measured by AYP percent 
proficient or by CST scale score). The effect 
was also stable over time (-0.24 to -0.33). 



At the elementary level, the AYP measure- 
percent proficient on CST in English language 
arts-is the only measure that favors charters 
(effect size of +0.05). The elementary school 
results are summarized in the table below. 

The API results are stable over time, 
though the differences in 2006 (as already 
stated, effect size of -0.10 or 10% of a stan- 
dard deviation) are more modest than in prior 
years. The API of charters trailed that of non- 
charters in 2004 (effect size of -0.22) and 
2002 (effect size of -0.20) by larger amounts. 



Summary of Elementary School Measures, Charter Versus Noncharter In 2006 



2006 Outcome Measure 


Average Score 
for Noncharters 
(nt=4,965) 


How Charters' Scores Differed (n=183) 


Charter Effect, After Adjusting 
for Enroilment and 
Schooi Characteristics^ 


Effect Size (in 
standard units) 


Base API 


766.1 


-2.1 API points means charters scored lower) 


■8.8 APi points*** 


■0.10 


AYP English, percent proficient 


46.5% 


+3.0 percentagG points** ("+" means charters scored higher) 


+1.2 percentage points* 


+0.05 


AYP Math, percent proficient 


55.6% 


■3.7 percentage points*** 


■5.2 percentage points*** 


■0.26 


CST English, Grade 3-mean scale score 


332.6 


-0.4 scale score points 


■2.9 scaie score points*** 


■0.10 


CST Math, Grade 3-mean scale score 


371.1 


-8.0 scale score points*** 


■10.0 scaie score points*** 


■0.26 


t In the tables throughout this report, “n” refers to the number of schools represented. 

£ How charters’ scores would have differed from noncharters' if charters had had the same enrollments and school characteristics as the noncharters. 
* Difference is significant at .10 ievel. ( 10 % chance that difference is due to random variation.) 

** Difference is significant at .05 level. ( 5 % chance.) 

••• Difference is significant at .01 level. ( 1 % chance.) 

Note: If no asterisk is present, the result is not statistically significant. 
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Charter middle schools significantly outperformed noncharter middle schools 



As is true in other states, charter middie 
schools are not very common in California. Only 
54 such schools, enrolling about 21,000 
students, are represented in the analyses 
below. All but one were classroom-based, and 
76% were start-ups. They are compared to 
1,211 traditional middle schools, which served 
about 1.15 million students in 2005-06. 

The two types of schools differ substantially 
in enrollment size-the median charterwas less 



than one-third the size of the median nonchar- 
ter. They also differed in the types of students 
served and teachers employed. As with elemen- 
tary schools, charter middle schools tended to 
have fewer English learners and fewer students 
whose most highly educated parent had not 
graduated from high school. However, the 
median charter middle school served more 
African American students and fewer white 
students than the median noncharter middle 



school. Charter middle schools also had fewer 
experienced and fully credentialed teachers. 
The overall level of challenge faced by charter 
middle schools was higher, as indicated by a 
lower median SCI value: 162.8 versus 165.3 
for noncharter middle schools. (Based on SCI 
values, the median charter middle school 
would be predicted to score 20 points lower 
on the API than the median noncharter mid- 
dle school.) 
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The rectangular boxes on the chart show the range from the 25th percentile (the bottom of the box) to the 75th percentile (the top of the box).The line inside the box marks the 50th percentile or the median (haif 
the schools are above this line and half are below).The small horizontal lines above and below the box mark the beginning of the “outliets'-those schools with enrollments (left chart) or with SCI values (right chart) 
far from the median. 



Both with and without controlling for differ- 
ences in enrollment and school characteristics, 
California’s middle school charters scored 
higher than noncharters on the API. After 
controlling for school characteristics, the effect 
is +40.9 points, or 44% of a standard devia- 
tion. The average API of a noncharter middle 
school is about 730, which is toward the 
bottom of Decile 6 on the API. The expected 



performance of a charter middle school that 
could somehow enroll “the same students” and 
employ "the same teachers” would be about 
771, toward the upper end of API Decile 7. 

This effect is substantial, and it is consistent 
across other measures as well. On all measures 
analyzed for middle schools, the difference 
between charter performance and noncharter 
performance is statistically significant at the 0.01 



level, meaning that there is less than a 1% chance 
that the difference between the two sets of schools 
was due to random variation. The middle school 
results are summarized in the table on page 11. 

The results are also stable over time. 
Adjusting for school characteristics, the API of 
charters substantially exceeds that of nonchar- 
ters in 2004 (effect size of +0.31) and 2002 
(effect size of +0.33). 
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Both with and without controlling for differences in enrollment and SCI, charter 
middle schools scored significantly higher than noncharters on the API. The results 
are consistent over time and across multiple measures. 



Summary of Middle School Measures, Charter Versus Noncharter in 2006 



2006 Outcome Measure 


Average Score 
for Noncharters 
(n=l,211) 


How Charters' Scores Differed (n=54) 


Charter Effect, After Adjusting 
for Enroiiment and 
Schooi Characteristics ‘ 


Effect Size (in 
standard units) 


Base API 


729.5 


+23.5 API points* (“+” means charters scored higher) 


+40.9 API points*** 


+0.44 


AYP English, percent proficient 


44.3% 


+4.4 percentage points* 


+8.7 percentage points*** 


+0.41 


AYP Math, percent proficient 


40.6% 


+3.6 percentage points 


+7.1 percentage points*** 


+0.34 


CST English, Grade 7-mean scale score 


342.5 


+10.3 scale score points*** 


+14.3 scaie score points*** 


+0.54 


CST Math, Grade 7-mean scale score 


339.9 


+9.1 scale score points** 


+14.8 scaie score points*** 


+0.50 


£ How charters’ scores would have differed from noncharters' if charters had had the same enrollments and school characteristics as the noncharters. 
* Difference is significant at .10 ievel. (10% chance that difference is due to random variation.) 

** Difference is significant at .05 level. (5% chance.) 

••• Difference is significant at .01 level. (1% chance.) 

Note: If no asterisk is present, the result is not statistically significant. 
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Results for charter high schools are positive but less consistent 



This report’s analyses reflect the performance 
of 109 charter high schools that served more 
than 68,000 students and 946 noncharter 
high schools that enrolled more than 1.7 million 
students. These charter high schools were 
generally start-ups (92%) and had a sub- 
stantial percentage of nonclassroom-based 
schools (38%). 

Charter high schools were very different 
from noncharter high schools in enrollment 



size, with the median charter less than one- 
fifth the size of the median noncharter. But 
when considering students’ ethnicity and 
parental education levels, charter high schools 
and their noncharter counterparts had more 
similarities than were found at the elementary 
and middle school levels. For example, the 
gap in the percentages of white and Latino 
students was relatively small. Differences in 
teacher experience and credential levels. 



however, were fairly sizable-just as they were 
at the other grade levels. When multiple chal- 
lenge factors are considered simultaneously, 
the typical charter high school seemed to face 
a greater level of challenge: the median char- 
ter’s SCI value was 156.8, and the median 
noncharter’s SCI was 162.7. (Based on SCI 
values, the median charter high school would 
be predicted to score 47 points lower on the 
API than the median noncharter high school.) 



2005-06 Enrollment for Charter and Noncharter 
High Schools Included in this Analysis 



2006 School Characteristics Index (SCI) Values for Charter 
and Noncharter High Schools Included in this Analysis 




The rectangular boxes on the chart show the range from the 25th percentile (the bottom of the box) to the 75th percentile (the top of the box). The line Inside the box marks the 50th percentile or the median (half 
the schools are above this line and half are below). The small horizontal lines above and below the box mark the beginning of the “outliets'-those schools with enrollments (left chart) or with SCI values (right chart) 
far from the median. 



After controlling for differences in enroll- 
ment and school characteristics, the study 
team found a mixture of favorable and unfa- 
vorable results for charter high schools. 

In API terms, charter high schools scored 
higher than noncharter high schools.The effect 
is +14.6 points, or 17% of a standard devia- 
tion. The average API of a noncharter high 
school was about 703, or just above the line 
that splits API Deciles 5 and 6. The expected 
performance of a charter high school that 



could somehow enroll “the same students” 
and employ "the same teachers” would be 
about 718, toward the upper end of API 
Decile 6. This effect is not large, but it is sta- 
tistically significant at the 0.01 level. 

Results on other measures suggest that the 
story in English language arts is different than 
in math. All of the high school performance 
results are statistically significant, and the API, 
English percent proficient (AYP), CST English 
scale score, and CAHSEE English scale score 



results all favor charters. But the two math in- 
dicators, percent proficient (AYP) and CAHSEE 
math scale score, favor noncharters. These 
results are summarized in the table on 
page 13. 

The results are not stable over time. The 
effect size of +0.17 pertaining to the 2006 API 
contrasts with a negligible effect favoring char- 
ters on the 2004 data (effect size of +0.01) 
and a substantial effect favoring charters on 
the 2002 data (effect size of +0.29). 
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After controlling for differences in enrollment and student characteristics, the study found 
that charter high schools scored higher than noncharters on the API, but the results were 
not stable over time. However, noncharter high schools outperformed charters in math. 



Summary of High School Measures, Charter Versus Noncharter in 2006 



2006 Outcome Measure 


Average Score 
for Noncharters 
(n=946) 


How Charters' Scores Differed (n=109) 


Charter Effect, After Adjusting 
for Enroiiment and 
Schooi Characteristics^ 


Effect Size (in 
standard units) 


Base API 


703.0 


-18.8 API points** means charters scored lower) 


+14.6 API points*** 

(“+” means charters scored higher) 


+0.17 


AYP English, percent proficient 


53.6% 


-5.4 percentage points**’* 


+3.0 percentage points** 


+0.13 


AYP Math, percent proficient 


49.7% 


■12.6 percentage points*** 


■5.1 percentage points*** 


-0.22 


CAHSEE English, Grade 10-mean scale score 


380.2 


-1.9 scale score points 


+4.3 scale score points* * * 


+0.26 


CAHSEE Math, Grade 10-mean scale score 


380.8 


-9.2 scale score points*** 


■2.6 scale score points*** 


-0.15 


CST English, Grade 10-mean scale score 


332.2 


-2.9 scale score points 


+5.7 scale score points*** 


+0.22 


£ How charters’ scores would have differed from noncharters' if charters had had the same enrollments and school characteristics as the noncharters. 

Difference is significant at .10 ievel. (10% chance that difference is due to random variation.) 

** Difference is significant at .05 level. (5% chance.) 

••• Difference is significant at .01 level. (1% chance.) 

Note: If no asterisk is present, the result is not statistically significant. 
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Charters run by management organizations generaiiy performed weii in 2006 



Some charter schools in California are founded 
by, and receive ongoing operational assistance 
from, nonprofit "charter management organiza- 
tions” (CMOs) or for-profit "educational 
management organizations” (EMOs).*^ This 
study sought to determine whether schools that 
are members of these organizations perform 
differently than those that are not members. 
(Most management organizations included in 
this analysis are nonprofit organizations.) 

CMOS and EMOs not only start and run 
charter schools, but they also develop curric- 
ula, share best practices, and streamline 
administrative costs. For this report, a 
CMO/EMO is defined as an organization, or 
branch of an organization, created to provide 
administrative support for multiple charter 
schools.^ However, this definition excludes 
all-charter districts. It also excludes 
community-based organizations or educational 
agencies created to serve a different or broader 



purpose but that also run one or more charter 
schools as part of that broader mission. Thus, 
the small handful of Conservation Corps offices 
and universities that support charter schools 
are not included as CMOs/EMOs in this analy- 
sis. Hereafter, charter schools belonging to a 
CMO or EMO are referred to as "member 
schools,” and the other charter schools are re- 
ferred to as "nonmember schools.” 

This analysis compares the performance of 
59 member schools serving about 29,000 
students to that of 287 nonmember schools 
serving about 138,000 students. The members 
were more likely to be middle and high schools 
than nonmembers were.*'* This is important to 
point out because, among all public schools 
statewide, achievement levels and the pace of 
improvement have varied among the three 
levels. Elementary schools have generally 
outperformed middle schools, which have 
outperformed high schools. However, results are 



not broken out by grade span because the small 
number of members at each level would yield 
results that are not as statistically meaningful 
as the results for the group as a whole. 

The median member school served greater 
percentages of students from traditionally 
lower-achieving groups. To take just one ex- 
ample, in the median member school, 17% of 
students had parents who have not graduated 
from high school, and the median nonmember 
school had only 7%. When a multitude of 
factors was considered to produce SCI values, 
member schools had a median value of 158.5, 
and nonmembers had a median SCI of 167.5. 
This means that the typical member school 
faced a markedly higher level of overall chal- 
lenge with respect to student test scores than 
did the typical nonmember school. Based on 
SCI values, the median member school would 
be predicted to score 72 points lower on the 
API than the median nonmember school. 
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The rectangular boxes on the chart show the range from the 25th percentile (the bottom of the box) to the 75th percentile (the top of the box). The line inside the box marks the 50th percentile or the median (half 
the schools are above this line and half are below). The small horizontal lines above and below the box mark the beginning of the “outliers"-those schools with enrollments (left chart) or with SCI values (right chart) 
far from the median. 
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After controlling for differences in enrollment and student characteristics, charters run 
by management organizations scored significantly higher than other charter schools. 



However, after controlling for differences 
in enrollment and student characteristics, 
member schools scored significantly higher 
than other charter schools. The effect was +40 
API points, or 40% of a standard deviation. This 
effect is substantial, and it holds across the 
AYP percent proficient measures in English 
and math, which apply at all school levels. 
(Because there are only 24 elementary, 14 



middle, and 21 high member schools, the 
numbers of students involved in single-grade 
test measures are relatively small. Therefore, 
results such as "grade 3 math” are less reli- 
able and not presented here. Member school 
performance was also examined by grade 
level, and the results were essentially un- 
changed. )The AYP percent proficient on CST in 
English language arts has an effect size of 



+0.29, and the AYP math measure has an 
effect size of +0.35. These outcomes are 
summarized in the table below. 

Many member schools have begun operat- 
ing very recently so results over time are not 
available for these schools. In general, growth 
in the number of CMO/EMO member schools 
has been strong in recent years, and continued 
growth is expected. 



Summary of Charter School Measures, CMO/EMO Member Schools Versus Nonmember Schools In 2006 



2006 Outcome Measure 


Average Score 
for Nonmembers 
(n=287) 


How CMO/EMO Members’ Scores 
Differed (n=59) 


CMO/EMO Effect, After Adjusting 
for Enroiiment and 
Schooi Characteristics^ 


Effect Size (in 
standard units) 


Base API 


738.6 


■8.8 API points (“■” means members scored lower) 


+39.5 API points*** {"+” means members 
scored higher) 


+0.40 


AYP English, percent proficient 


49.5% 


-3.1 percentage points 


+6.4 percentage points*** 


+0.29 


AYP Math, percent proficient 


46.4% 


-1.9 percentage points 


+8.2 percentage points*** 


+0.35 


£ How CMO/EMO members’ scores would have differed from nonmembers’ scores if CMO/EMO members had had the same enrollments and school characteristics as the nonmembers. 
* Difference is significant at .10 level. (10% chance that difference is due to random variation.) 

*'* Difference is significant at .05 level. (5% chance.) 

••• Difference is significant at .01 level. (1% chance.) 

Note: If no asterisk is present, the result is not statistically significant. 
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A comparison of conversions and start-ups yieids mixed resuits 



When charter schools began operating in Cali- 
fornia in 1993, most were conversions of 
existing schools.Today, however, a large major- 
ity are start-ups. Among the charters included 
in this analysis, three-quarters fit in that latter 
category. 

In 2005-06, the two types of charters 
differed from each other in the kinds of students 
they served, but the differences were not very 
large. In contrast, conversions and start-ups had 
notable differences in their teaching staffs, with 



conversions having more experienced and fully 
credentialed teachers. With respect to school 
size, conversions tended to be bigger than start- 
ups: the median conversion had an enrollment 
that was about two-thirds larger than the 
median start-up’s enrollment. Regarding overall 
challenge levels, conversions tended to face a 
lower level of challenge, as indicated by their 
higher median SCI value: 169.6 versus 164.6 
for start-ups. (Based on SCI values, the median 
conversion charter school would be predicted to 



score 40 points higher on the API than the 
median start-up charter school.) 

When considering the performance of the 
two types of charter schools, it is important to 
remember that conversions and start-ups had 
different compositions of elementary, middle, 
and high schools. For example, 11% of conver- 
sion charters were high schools, and 38% of 
start-ups served those grades. The perform- 
ance analyses on page 17 are reported by 
grade level to better compare like with like. 



2005-06 Enrollment for Conversion and 
Start-up Charter Schools Included in this Analysis 



2006 School Characteristics Index (SCI) Values for Conversion 
and Start-up Charter Schools Included in this Analysis 




The rectangular boxes on the chart show the range from the 25th percentile (the bottom of the box) to the 75th percentile (the top of the box). The line inside the box marks the 50th percentile or the median (half 
the schools are above this line and half are below). The small horizontal lines above and below the box mark the beginning of the “outliers"-thosesohools with enrollments (left chart) or with SCI values (right chart) 
far from the median. 
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With the exception of elementary math performance, start-up charter schools did 
not score significantly differently than conversion charters, after controlling for 
differences in enrollment and student characteristics. 



With the exception of eiementary math 
performance, start-up charter schoois did not 
score significantiy differentiy than conversion 
charters, after controiiing for differences in 
enroiiment and schooi characteristics. Looking 
across various measures yieidssmaii numbers 
of schoois for many of the anaiyses, and the 
results are usually not statistically significant. 
Elementary math performance on the CST is 
the exception, with start-up charters scoring 
11.2 scale score points lower than conversions 



(for an effect size of -0.27). This lower CST 
performance is also reflected in the AYR meas- 
ure, for which the average percentage profi- 
cient is 7.3 percentage points lower at start-up 
charters (effect size of -0.36). The results for 
elementary conversions and start-ups are 
summarized in the table below, but results for 
middle and high schools are not included 
because none was statistically significant.^ 
Similar to the results from 2006, differ- 
ences in previous years were generally not 



statistically significant. However, start-ups’ 
lower CST math performance at the elemen- 
tary level appears in the historical data. (In 
addition to the analyses of start-ups and 
conversions separated by grade span [elemen- 
tary, middle, and high], the study team 
compared the performance of all start-ups to 
all conversions without separating them by 
grade but using a statistical adjustment for 
grade level. The results were similar.) 



Summary of Start-up and Conversion Charter School Measures in 2006 (Elementary Schools Only) 



2006 Outcome Measure 


Average Score 
for Conversions 


How Start-ups' Scores Differed 


Start-up Effect, After Adjusting 
for Enrollment and 
School Characteristics^ 


Effect Size (in 
standard units) 




Elementary Only, Conversion (n=62) and Start-up (n=121) 






Base API 


772.8 


-13.3 API points means start-ups scored lower) 


-9.7 API points 


-0.11 


AYP English, percent proficient 


48.8% 


+1.1 percontago points (“+” means start-ups scored higher) 


+2.9 percentage points 


+0.13 


AYP Math, percent proficient 


56.9% 


-7.5 percentage points’*’*’* 


-7.3 percentage points’*’*’* 


-0.36 


CST English, Grade 3-mean scale score 


332.8 


-1.1 scale score points 


+0.9 scale score points 


+0.03 


CST Math, Grade 3-mean scale score 


371.6 


-12.9 scale score points’*’* 


-11.2 scale score points’* 


-0.27 


£ How start-up charter schools’ scores would have differed from conversion charters if start-ups had had the same enrollments and school characteristics as the conversions. 
* Difference is significant at .10 level. (10% chance that difference is due to random variation.) 

*'* Difference is significant at .05 level. (5% chance.) 

••• Difference is significant at .01 level. (1% chance.) 

Note: If no asterisk is present, the result is not statistically significant. 
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Classroom-based charters outperformed nonclassroom-based charters, after controlling for student 
characteristics and enrollment 



Among the charter schools represented in 
this report, four out of five are considered 
classroom-based, with the remainder meeting 
the State Board of Education’s definition of 
"nonclassroom-based.” A charter school is 
considered nonclassroom-based when at least 
20% of its instructional time does not involve 
students on site under the direct supervision of 
a teacher. Schools that provide a substantial 
portion of their instruction through home 
schooling, independent study, or distance 
learning (instruction via Internet-connected 
computers) generally fit that definition, as do 
schools that rely heavily on community-based 
learning through internships and field trips. 

Classroom- and nonclassroom-based char- 
ters differed substantially in the types of 
students they served, with nonclassroom- 
based students more likely to be white and 



have at least one college-educated parent and 
less likely to be Latino or an English learner. 
Nonclassroom-based charters also tended 
to have fewer teachers with two years of 
experience or less. It is therefore somewhat 
surprising that nonclassroom-based charters 
had a lower median SCI value (higher overall 
challenge level): 161.2 versus 166.6 for 
classroom-based charters. (Based on SCI 
values, the median nonclassroom-based charter 
school would be predicted to score 43 points 
lower on the API than the median classroom- 
based charter school.) 

Perhaps it is nonclassroom-based charters’ 
relatively high student mobility rate, which 
tends to harm academic achievement, that 
brings down their SCI values. In the median 
nonclassroom-based charter, only 79% of 
students who were enrolled during spring 



testing had been enrolled since the prior Octo- 
ber.That compares negatively with the 95% rate 
in the median classroom-based charter school. 

As with conversions and start-ups, 
classroom- and nonclassroom-based charters 
differ in their elementary, middle, and high 
school make-up and therefore their perform- 
ance results are reported by grade-level type.“ 

It is also important to note that 
nonclassroom-based charters may tend to 
serve different types of students even when 
comparisons are confined to schools serving 
the same grade span. For example, some 
nonclassroom-based schools are designed for 
students who have not succeeded in the 
traditional school setting and serve as tempo- 
rary transitions back into more mainstream 
schools. In addition, other nonclassroom-based 
charters are networks of home schoolers. 
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The rectangular boxes on the chart show the range from the 25th percentile (the bottom of the box) to the 75th percentile (the top of the box). The line inside the box marks the 50th percentile or the median (half 
the schools are above this line and half are below). The small horizontal lines above and below the box mark the beginning of the “outliers"-thosesohools with enrollments (left chart) or with SCI values (right chart) 
far from the median. 
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Among charter elementary schools, nonclassroom-based charters scored lower than 
classroom-based charters on the 2006 API, after adjusting for differences 

in enrollment and student characteristics. 



Nonclassroom-based charters scored lower 
than classroom-based charters on the 2006 API, 
after adjusting for differences in enrollment and 
school characteristics. The effect held for 
elementary schools (about -19 points, or 22% of 
a standard deviation) as well as for high schools 
(about -14 points, or 14% of a standard devia- 
tion). No results are reported for middle schools 
because there is only one nonclassroom-based 



Data: California Department of Education (CDE) 



school. The effects are small, and only the 
elementary result is significant at the 0.10 level. 
For all but one of the other performance results, 
nonclassroom-based charters score lower, with 
much larger gaps in math. For the elementary 
AYP indicator for English language arts, 
nonclassroom-based charters outscore 
classroom-based charters by about seven 
percentage points (effect size of +0.33). These 



results are summarized in the table below. (In 
addition to analyses of classroom-based charters 
and nonclassroom-based charters separated by 
grade span [elementary, middle, and high], the 
study team compared the performance of all 
classroom-based charters to all nonclassroom- 
based charters without separating them by 
grade but using a statistical adjustment for 
grade level. The results were similar. ) 
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Summary of Nonclassroom-based and Classroom-based Charter School Measures In 2006 



2006 Outcome Measure 


Average Score for 
Classroom-based 
Charters 


How Nonclassroom-based Charters' 
Scores Differed 


Nonclassroom-based Effect, 
After Adjusting for Enrollment 
and School Characteristics^ 


Effect Size (in 
standard units) 


Elementary School Only, Classroom-based (n=156) and Nonclassroom-based (n=27) 


Base API 


767.0 


-20.0 API points (‘‘-"means that nonclassroom-based 
charters scored lower) 


-19.3 API points* 


-0.22 


AYP Englisli, percent proficient 


48.5% 


■^6.5 percentage points*** fv means that 
nonclassroom-based charters scored higher) 


+7.2 percentage points*** 


+0.33 


AYP Math, percent proficient 


53.6% 


-11.3 percentage points*** 


■12.4 percentage points*** 


-0.62 


CST English, Grade 3-mean scale score 


332.5 


-2.6 scale score points 


■2.4 scaie score points 


-0.08 


CST Math, Grade 3-mean scale score 


367.2 


-28.2 scale score points*** 


-28.5 scaie score points*** 


-0.69 


Middle School Only, Classroom-based (n=53) and Nonclassroom-based (n=l)* 


High School Only, Classroom-based (n=68) and Nonclassroom-based (n=41) 


Base API 


700.7 


-43.9 API points** 


■13.6 APi points 


-0.14 


AYP English, percent proficient 


51.1% 


-7.6 percentage points** 


■1.8 percentage points 


-0.08 


AYP Math, percent proficient 


42.7% 


-14.9 percentage points*** 


■11.3 percentage points*** 


-0.52 


CST English, Grade 10-mean scale score 


334.9 


-14.8 scale score points** 


■5.8 scaie score points 


-0.20 


CAHSEE English, Grade 10-mean scale score 


381.7 


-8.9 scale score points** 


■3.2 scaie score points 


-0.18 


CAHSEE Math, Grade 10-mean scale score 


377.1 


-14.5 scale score points*** 


■9.6 scaie score points*** 


-0.53 


£ How nonclassroom-based (NCB) charter scores would have differed from classroom-based (CB) charters’ scores if NCBs had had the same enrollments and school characteristics as the CBs. 
¥ Results are not shown for middle schools because only one middle school was nonclassroom-based. 

* Difference is significant at .10 level. (10% chance that difference is due to random variation.) 

Difference is significant at .05 level. (5% chance.) 

••• Difference is significant at .01 level. (1% chance.) 

Note: If no asterisk is present, the result is not statistically significant. 
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Unadjusted API Results 

Basic unadjusted performance statistics aiiow for comparisons to previous EdSource reports 



In past reports on charter school perform- 
ance, EdSource did not perform regression 
analyses to “adjust” outcomes based on 
school characteristics. Basic, unadjusted 
API data can facilitate comparisons to 
those past EdSource reports, and the 
growth target information shows how well 
charter versus noncharter schools improved 
from 2005 to 2006. 

The API is organized in two-year 
cycles. The first year of the cycle is 
the “Base” year, against which scores 
from the second (“Growth”) year are 



compared. In the Base API, schools of 
the same type (elementary, middle, or 
high) are ranked in 10 bands, called 
deciles, with each decile representing 
10% of schools. This means that about 
30% of schools overall occupy the three 
high-performing deciles, 40% fall in the 
mid-performing deciles (4—7), and the 
remaining 30% are in the three low- 
performing deciles. Schools are ranked 
in two ways — against all others in the 
state and against the 100 schools with 
the most similar SCI values. 



Charters' performance on the 2006 Base 
API Statewide and Similar Schools rankings 
echo the main findings in this report 

Charters’ Statewide Rankings show that 
elementary charter schools as a whole 
performed similarly to the entire set of 
California’s elementary schools. The 
percent of schools in the low, middle, 
and high performance ranges were simi- 
lar to the statewide distribution of 30%, 
40%, and 30%, respectively. In contrast, 
the set of charter middle schools had 
better absolute performance than the 
state’s middle schools as a whole: the 
distribution of charter middle schools 
among the low/middle/high ranges 
skews toward the high range. The oppo- 
site is the case with charter high schools, 
with their distribution skewing toward 
the low range. The upper set of bars in 
Figure 2 shows the exact percentages. 
The dotted lines mark the distribution 
for the entire state. 

In contrast to the Statewide Rank- 
ings, which show schools’ absolute 
performance, the Similar Schools Rank- 
ings take into account the overall level of 
challenge that schools face and rank 
schools’ performance against those with 
roughly the same level of challenge (as 
measured by SCI values). The results are 
displayed in the lower set of bars in 
Figure 2. The similar schools results are a 
restatement of what was shown earlier in 
the regression results in that they show 
performance relative to schools’ SCI 
values. Elementary charters had a dis- 
proportionate share of schools that 
performed relatively poorly, given their 
school characteristics, which pulled the 
group’s average performance down. On 
the other hand, charter middle and high 
schools had relatively large portions of 
schools that scored well, which produced 
a positive “charter effect” on the API. 



Charters’ Statewide API Rankings in 2006 show that elementary charters as a whole performed 
similarly to noncharter elementary schools, but charter middle schools ranked higher than their 
noncharter counterparts. However, regular high schools outdid charter high schools. 



figure 2 I Charter Schools’ 2006 Base API Statewide and Similar Schools Rankings 



I Low (Deciles 1-3) ■ Middle (Deciles 4-7) High (Deciles 8-10) 




0 % 10 % 20 % 30 % 40 % 50 % 60 % 70 % 80 % 90 % 100 % 

Percent of Schools 

The dotted lines show the statewide distribution of all schools-including charters-on the API Rankings, which is 30% (low), 40% 
(middle), and 30% (high). The upper set of bars shows the percentages of charter schools in the three ranges of the Base API Statewide 
Rankings. The lower set is based on the Similar Schools Rankings. 



Data: California Department of Education 
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Charters performed well on the 2006 
Growth API 

The Growth API indicates whether 
schools have met state-set goals for 
improvement from one year to the next. 
In the 2005 Base API/2006 Growth 
API cycle, schools were expected to 
improve their schoolwide API scores by 
5% of the difference between their 
Base score and 800, the state’s official 
goal. (Schools with API scores of 800 
and above were expected to keep their 
scores at 800+.)'^ In addition, each 
“numerically significant subgroup” of 
students had a growth target that 
was 80% of their schools’ targets. 
(Subgroup growth targets have since 
become more rigorous.*®) 

While Statewide Rankings show ab- 
solute performance and Similar Schools 
Rankings show performance relative to 
schools with similar levels of challenge, 
the percent of schools meeting API 
growth targets shows how groups of 
schools are progressing. 

In the 2006 Growth API, charters 
overall met both their schoolwide and 
subgroup targets in higher percentages 
than noncharters. (See Figure 3.) This 
was true at all three grade levels. The 
pattern in 2006 was quite similar to 
what it has been in the last two API 
cycles, except that in 2005, charter 
high schools were less successful than 
noncharters. 

When comparing the Growth API 
results of different types of charters, 
the biggest difference is between 
classroom-based charters and nonclassroom- 
based charters, with the former more 
likely to meet their growth targets (64% 
to 44%). CMO/EMO charters are also 
more likely than nonmembers (67% to 
59%) to meet their growth targets. The 
difference between start-ups and conver- 
sions (61% to 59%) is the smallest. 
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The 2006 Similar Schools Rankings shows that elementary charters had a disproportionate 
share of schools that performed relatively poorly, given their student characteristics. 
On the other hand, charter middle and high schools had relatively large portions 
of schools that scored well, which produced a positive “charter effect” on the API. 



figure 3 I 2006 Growth API Results 



Charter vs. Noncharter 




Charters of Different Types 




Data: California Department of Education (CDE) EdSource 6/07 



June 2007 • California Charter School Performance • 21 



EDSOURCE REPORT 



This 2007 EdSource report replicates some analyses done previously by EdSource in its 2005 and 2006 charter 
performance updates, but extends and deepens the analysis 



In 2005 and 2006, EdSource issued reports on California charter schools’ 
academic performance. The reports described the performance of charters and 
noncharters in terms of Base API Statewide and Similar Schools rankings and 
the percentage of schools meeting API growth targets. EdSource also examined 
performance by different types of charters-conversion versus start-up and 
classroom- versus nonclassroom-based. Unlike the current report, neither of the 
prior reports used statistical regression (see pages 4-5) to control for school 
characteristics. 

The 2005 EdSource Study 

Key findings, based on 2004 performance data, included: 

• At all levels-elementaty, middle, and high-a greater percentage of charter 
schools than noncharter schools met their API growth targets. 

• Charter elementary and high schools had mixed success on Base API 
Statewide and Similar Schools rankings. 

• Charter middle schools had a strong showing on both rankings. 

• A greater percentage of conversion charters met their growth targets than 
start-ups. 

• Classroom-based charters were much more likely to meet growth targets than 
nonclassroom-based charters. 

• Charter schools that had been established for two or more years were slightly 
less likely to meet their targets than newer charters. 

• On both the math and English portions of the CAHSEE, charters lagged 
noncharters in the percentage of 10th graders who passed. 



The study also compared the improvement of charter and noncharter schools on 
California Standards Tests in math and English, matching subsets of charter and 
noncharter schools. Charters generally showed greater progress than noncharters 
in both increasing the percentage of students scoring “proficient” and above and 
decreasing the percentage scoring “far below basic.” But these results may have 
reflected the performance of quite different sets of students. 

Einally, that study provided a summary of some California- and nationally-based 
performance analyses done by other researchers. 

The 2006 EdSource Study 

This report repeated much of the previous year’s analyses with 2005 data. The 
main findings included: 

• Charter elementary schools were more likely to meet their growth targets than 
noncharters were, but had Base API Statewide Rankings that were similar. 

• Among middle schools, charters again excelled in both meeting growth targets 
and Base API Statewide Rankings. 

• Charter high schools were slightly less likely to meet their growth targets than 
noncharter high schools were and lagged noncharters in the Base API 
Statewide Rankings. 

• Conversions were more successful in meeting API growth targets than were 
start-ups, and classroom-based charters outperformed their nonclassroom- 
based counterparts on the same measure. 

• On the CAHSEE, charter 10th graders again passed the CAHSEE in lower 
percentages than noncharter 10th graders. 



Limitations and Conclusion 



This analysis has limitations 

Measuring and comparing the perform- 
ance of schools is always a complicated 
endeavor. Determining what aspects of 
those schools might be contributing to 
performance is even more challenging. 
But as California struggles to improve the 
performance of its public schools and its 
students — and as some look to the states 
growing charter school movement to 
contribute — it is important to regularly 
and impartially examine whether, to what 
extent, and in what ways charters are 
making such a contribution. 

This EdSource report compares the 
performance of charter schools to that of 
noncharters and the performance of types 



of charter schools to other types. To make 
certain the conclusions of this work are as 
sound as possible, this analysis put particular 
emphasis on controlling for the measurable 
student characteristics that are most strongly 
related to school performance. 

However, like all analyses, this one 
also has its limitations. For example, as 
has been pointed out, the report does not 
account for the motivation level of 
students, which may differ between 
noncharters and charters, whose students 
attend because their parents have chosen 
to send them there. (On the other hand, 
that choice is sometimes made because 
the student is not succeeding in the regu- 
lar public school system.) 



Nor does this analysis account for 
differences in schools’ resources because 
school-level data on finances is not avail- 
able. Just as traditional public schools 
differ in the resources they have available 
to them, so do charter schools, with some 
charters struggling with start-up funding 
and facilities challenges and others 
supported by private philanthropic 
contributions that allow for a longer 
school day and year. 

In addition, this analysis does not 
look for differences in the academic 
performance of schools that may be 
related to how long they have been in 
existence. Some say that assessing the per- 
formance of a school that has operated as 
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• I To Learn More 

Information on Charter School Laws and Policies 

See the charter school section of EdSource Online for an overview, relevant data, and a list of EdSource 
publications related to charter schools: www.edsource.org/edu_chart.cfm 

The California Department of Education (CDE) also provides a great deal of information on its website: 

www.cde.ca.gov/sp/cs/re/ 

Data About Individual Charter Schools in California 

The Ed-Data Partnership website-www.ed-data.kl2.ca.us-provides a wealth of data about every char- 
ter school in California, including student background, staffing information, and summary adequate yearly 
progress and Academic Performance Index reports. Data from as far back as 1992-93 are available. 

The GreatSchools website-www.greatschools.net-provides free profiles of all California schools with 
performance, student, and teacher data. 

Charter School Organizations 

See the California Charter Schools Association's website: www.myschool.org 
See the Charter Schools Development Center’s website: www.cacharterschools.org 

Other Research on Charter School Performance 

To learn more about the quality of research on the academic performance of charter schools, as well as 
recommended analytic methods, go to the National Charter School Research Project’s website at: 

www.ncsrp.org/ 



a charter for only a year or two is not fair 
or is more an assessment of the schools 
that its students previously attended. 

Furthermore, this analysis does not 
compare charter schools only to the nearby 
public schools serving similar grade spans, 
which would better speak to the choices that 
parents actually face in their communities. 
Finally, this analysis is limited to standard- 
ized measures of academic performance 
and does not consider other outcomes that 
may be important to educators and parents. 

Performance comparisons are mixed, but 
some findings stand out as intriguing 

Using statistical controls for differences in 
enrollment and student characteristics 
summarized by the SCI, this analysis of 
performance results for California schools 
on the 2006 Base API and other measures 
yielded several interesting findings. 

In comparisons of charter to noncharter 
schools, charter middle schools look 
especially strong 

As a group, charter elementary schools 
had lower API scores than traditional 
public schools, but charter middle and 
high schools generally scored higher. For 
the findings generally, the difference 
between charters and noncharters tended 
to be statistically significant, but the 
effect sizes were not large, especially at the 
elementary and high school levels. 

Flowever, among middle schools, char- 
ters outperformed noncharters on all 
measures by a statistically significant margin, 
and the effect sizes were larger. (Tests of 
statistical significance take into account the 
fact that the state has a relatively small 
number of charter middle schools.) 

This strong performance by charter 
middle schools has been stable for several 
years. Some important questions for Cali- 
fornia include: what is happening in these 
charter schools, in what ways do they 
differ from other middle schools, and are 
there some lessons that emerge from these 
differences that could help California 



improve education in traditional public 
schools at the middle grades? 

Perhaps some clues to charter middle 
schools’ strong performance can be found 
in the fact that 26% of them were 
members of management organizations, 
which generally produced positive results. 

Classroom-based charters and members 
of management organizations showed 
stronger performance 

Classroom-based charter schools gener- 
ally outperformed nonclassroom-based 
charters at both the elementary and high 
school levels. Differences in math were 
statistically significant and larger than 
most differences found in this study. 

Statistically significant differences 
in math performance were also found 
between elementary conversion and start- 
up charters, with conversions coming out 
on top. Otherwise, conversions and start- 
ups performed fairly similarly. 

In contrast, charter schools that were 
members of CMOs/EMOs scored higher 



than nonmembers by a statistically significant 
margin on all three performance measures 
examined — ^API scores and AYP measures 
of percent proficient in math and English, 

Educators and policymakers may 
benefit from learning more about what 
CMOs/EMOs are doing to achieve 
reasonably strong results with their 
students, many of whom are low-income. 

The link between “autonomy” and 
performance bears closer examination 

The question of whether and how 
increased school autonomy leads to 
improved performance is an important one 
that likely deserves more study based on a 
nuanced understanding of the concept. 

For a charter school, this concept of 
“autonomy” can mean freedom from 
constraints, but it can also mean a lack of 
external support. With regard to resources 
and finances, freedom can take the form of 
receiving funds directly from the state 
rather than through the chartering agency, 
being able to spend “categorical” 
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(earmarked) dollars as the school chooses, 
or not having to comply with most of the 
states education laws. On the other hand, 
“autonomy” can mean having to find and 
pay for facilities, professional development 
for staff, and administrative functions such 
as payroll and data reporting. 

With regard to the instructional 
program, charter school autonomy might 
mean the school can choose its program 
and approach, and then attract students 
and teachers for whom it is a good fit. On 
the other hand, instructional autonomy 
could mean that the school has difficulty 
finding appropriate sources for teacher 
professional development or textbooks. 
And within the charter community itself 



“school autonomy” to determine an 
instructional program can vary widely 
from networks of home-schooled fami- 
lies with great freedom to determine what 
is taught to CMO/EMO members. 
With some CMOs/EMOs there is free- 
dom to be different from traditional 
public schools but not much freedom to 
vary the instructional program and 
approach within that network. 

This 2007 EdSource analysis indicates 
that charter schools run by management 
organizations tend to achieve relatively 
strong academic results. These schools may 
have the “best of both worlds” in that they 
have a good mix between the freedoms or 
flexibility that charter schools are granted 



and the external support that most good 
districts provide for noncharter schools. 
These schools also likely benefit from 
having some level of shared vision within 
each management organization, an 
ability to attract students and teachers 
who subscribe to that vision, and an oppor- 
tunity to share best practices with a network 
of schools. 

As California policymakers and 
school reformers continue to explore the 
ways that charter schools can improve 
student achievement and spark innova- 
tion in the entire education system, the 
role and impact of autonomy — in both 
charter and noncharter schools — bears 
further examination. QQ 



Endnotes 

1 See Section 47607 of California’s Education Code to learn more about the conditions 
and procedures of charter revocation and renewal. 

2 The analytic approach of this report is informed by a study from the National Charter 
School Research Project (NCSRP), Key Issues in Studying Charter Schools and 
Achievement (Betts & Hill, 2006). 

3 Tests for “stability” involved analyses on 2006 data as well as data at two additional 
points in time, based on the availability and reliability of the data. Academic 
Performance Index and California StandardsTest scores were examined at 2006, 2004, 
and 2002. Adequate yearly progress and California High School Exit Exam data were 
examined at 2006, 2005, and 2004. 

4 For more details on the criteria for inclusion, see theTechnical Appendix on the EdSource 
website: www.edsource.org/pdf/CharterSchoolTechApp.pdf 

5 More on the ASAM is available on the California Department of Education’s website: 

www.cde.ca.gov/ta/ac/am/ 

6 See Technical Appendix at www.edsource.otg/pdf/CharterSchoolTechApp.pdf for more 
information on how the California Department of Education classifies schools as elemen- 
tary, middle, or high for the API. 

7 Students with significant cognitive disabilities who are unable to take the California 
Standards Tests take the California Alternate Performance Assessment (CAPA). CAPA scores 
play a part in the Academic Performance Index and adequate yearly progress measures. 

8 A “norm-referenced” test is one with scores that are expressed in terms of a student’s 
performance relative to a nationally representative sample of students. 

9 Using these grade-subject combinations is consistent with past EdSource charter 
reports, reflects policymakers’ identification of these as benchmark grades, includes a 
direct writing sample at grade 7, allows for examination of comparable mean scale 
scores, and avoids complications arising from the fact that schools differ in the percent- 
age of students enrolled in various math classes beginning with grade 8. 



10 In this report, tests of statistical significance are usually based on a t-test applied to a 
regression coefficient. 

11 Schools are usually the unit under study in this report. Thus, an effect size of 0.50, for 
example, indicates an effect that is about half as large as the typical variation in school- 
level scores. Effect sizes are often reported in the context of individual-level scores. Because 
our data are at the school level and may not even exist at an individual level (e.g., API), it 
makes sense to compare our findings to the school-level standard deviation. The reader 
should be cautious, however, because guidelines for interpreting effect size vary and 
usually are based on individual-level distributions. 

12 Some people use “CMO” and “EMO” interchangeably as umbrella terms for both nonprofit 
and for-profit management organizations. 

13 To see a list of management organizations and schools represented in this study, see 
theTechnical Appendix on the EdSource website: 

www.edsource.org/pdf/CharterSchoolTechApp.pdf 

14 Among member schools, 41% were elementary, 24% were middle, and 36% were 
high schools. In contrast, 55% of nonmember schools were elementary, 14% were 
middle, and 31% were high schools. 

15 The 2006 results for conversion and start-up charters at all three levels (elementary, 
middle, and high) can be found in the Technical Appendix on the EdSource website: 

www.edsource.org/pdf/CharterSchoolTechApp.pdf 

16 Among classroom-based charters, 56% were elementary, 19% middle, and 25% high 
schools. In contrast, nonclassroom-based charters had the following percentages: 39% 
elementary, 2% middle, and 59% high school. 

17 Academic Performance Index (API) scores can range from 200 to 1,000. 

18 To see how the rigor of API growth targets has been increased, go to: 

www.cde.ca.gov/ta/ac/ap/documents/rnfoguide 06 b.pdf 
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